On Reliable and Efficient Simulation for Possibly Ill-behaved Econometric Posteriors: Some Experiments with Neural Network Sampling with Applications to Iv Models, Mixture Processes and Option Evaluations

It is shown that artificial neural networks may serve as perfect candidate/ importance densities in Markov chain Monte Carlo and Importance Sampling methods for two reasons. First, artificial neural network functions possess a universal approximation property. Second, it is also easy to sample pseudo random draws from such networks. Given this existence property, several procedures are presented to search for such neural networks. First, it is shown that an analytical approach involving the construction of a perfect neural network exists. In this approach, use is made of a multi-layer perceptron with arctangent activation function. This neural network falls within the class for which approximation capabilities have been derived by Hornik, Stinchcombe and White (1989). For this type of neural network function when considered as a density kernel (on a bounded domain) all moments can be evaluated by analytical integration. However, the construction of such an analytical approximation requires usually (too) much time. A (usually much) quicker alternative is given by a simulation approach that uses a mixture of Student t densities as a candidate density for the posterior density of the parameters of interest. The methods are applied to a IV model with reduced rank (for the data of Angrist and Krueger (1991) on education and income), a two-regime dynamic mixture process for US real GNP, and to an method for efficient evaluation of options. The results compare favorably with other simulators such as Gibbs sampling with data augmentation and the Metropolis-Hastings algorithm or IS with a normal or t candidate.